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(Dated: August 31, 2006) 

We present a search for the production of a new heavy gauge boson W' that decays to a top 
quark and a bottom quark. We have analyzed 230 pb _1 of data collected with the D0 detector 
at the Fermilab Tevatron collider at a center-of-mass energy of 1.96 TeV. No significant excess of 
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events above the standard model expectation is found in any region of the final state invariant mass 
distribution. We set upper limits on the production cross section of W bosons times branching 
ratio to top quarks at the 95% confidence level for several different W' boson masses. We exclude 
masses between 200 GeV and 610 GeV for a W' boson with standard-model-like couplings, between 
200 GeV and 630 GeV for a W' boson with right-handed couplings that is allowed to decay to both 
leptons and quarks, and between 200 GeV and 670 GeV for a W' boson with right-handed couplings 
that is only allowed to decay to quarks. 

PACS numbers: 13.85.Rm; 14.70.Pw 



The top quark sector offers great potential to look for 
new physics related to electroweak symmetry breaking. 
In particular, it is a sensitive probe for the presence of 
additional gauge bosons beyond those of the standard 
model (SM). Such new gauge bosons typically arise in 
extensions to the SM from the presence of additional 
symmetry groups Q, Q . 

Direct searches for the production of additional heavy 
gauge bosons have focused on the lepton final state of the 
W boson decay which has good separation between the 
W boson signal and the SM backgrounds. The W bo- 
son lower mass limit in this decay channel is 786 GeV Q . 
In these studies, the W boson is allowed to have right- 
handed interactions with leptons and quarks, and it is 
assumed that the right-handed neutrino is lighter than 
the W boson. It is also possible that such a W boson 
does not interact with leptons and neutrinos but only 
with quarks. Searching in the quark decay channel avoids 
assumptions about the mass of a possible right-handed 
neutrino. Previous direct searches for W bosons in the 
quark decay channel have excluded the mass range be- 
low 261 GeV H and between 300 GeV and 420 GeV 0. 
Assuming that the W boson decays only to quarks and 
not to leptons yields a lower mass limit of 800 GeV 0. 
A search has also been performed in the single top quark 
final state of the VF'-boson decay. Assuming the W bo- 
son has only right-handed interactions and does not de- 
cay to leptons, the lower limit on the W boson mass is 
566 GeV 0. The comprehensive search presented here 
includes all of these W boson models. Indirect searches 
for evidence of a W boson depend on exactly how it in- 
terferes with the SM W boson and the results are thus 
highly model specific (see Ref. and references therein). 

The top quark was discovered in 1995 by the CDF 
and D0 collaborations 0, but the production of sin- 
gle top quark has not yet been observed. Both col- 
laborations have searched for single top quark produc- 
tion UGH EHH El At the 95% confidence level, the 
upper limit measured by D0 on the s-channel process is 
6.4 pb, and the limit measured by CDF is 13.6 pb. At 
the same confidence level, the limit on the t-channel pro- 
duction cross section is 5.0 pb from D0 and 10.1 pb from 
CDF. For comparison, the next-to-leading order (NLO) 
SM single top quark production cross sections are 0.88 pb 
in the s-channel and 1.98 pb in the i-channel [l^ . 

The single top quark final state is especially sensitive 



to the presence of an additional heavy boson, owing to 
the decay chain W — > tb, where the top quark decays to 
a b quark and a SM W boson. This decay is kinematically 
allowed as long as the W 1 mass is larger than the sum of 
top and bottom quark masses, i.e. as long as it is above 
about 200 GeV. 

An additional heavy boson would appear as a peak in 
the invariant mass distribution of the tb final state. Note 
that in this letter, the notation tb includes both final 
states W + — > tb and W ~ — > tb. The leading order 
Feynman diagram for W boson production resulting in 
single top quark events is shown in Fig.^ This diagram 
is identical to that for SM s-channel single top quark 
production where the SM W boson appears as the virtual 
particle [3 Hill E3. 




FIG. 1: Leading order Feynman diagram for single top quark 
production via a heavy W' boson. The top quark decays to 
a SM W boson and a b quark. 

The W' boson also has a i-channel exchange that leads 
to a single top quark final state. However, the cross sec- 
tion for a ^-channel W process is much smaller than the 
SM i-channel single top quark production due to the high 
mass of the W boson. It will thus not be considered in 
this letter. 

The SM W boson from the top quark decay then 
decays leptonically or hadronically. A heavy W' bo- 
son could also contribute to the top quark decay, but 
that contribution is negligible, again because of the large 
W' boson mass, and will not be considered here. 

We investigate three models of W boson production. 
In each case, we set the CKM mixing matrix elements for 
the W boson equal to the SM values. In the first model 
{W' L ), we make the assumption that the coupling of the 
W boson to SM fermions is identical to that of the SM 
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W boson. Under these assumptions, there is interference 
between the SM s-channel single top quark process and 
the W boson production process from Fig. ^ This in- 
terference term is small for large W boson masses, but it 
becomes important in the invariant mass range of a few 
hundred GeV where the SM s-channel production cross 
section is largest. In our modeling of the W' boson pro- 
duction process, we take this interference into account. 
This is the first direct search for W boson production to 
do so. 

In the second and third model (W R ), the W boson 
has only right-handed interactions, hence there is no in- 
terference with the SM W boson. In the second model, 
the W R boson is allowed to decay both to leptons and 
quarks, whereas in the third model it is only allowed to 
decay to quarks. The main difference between these two 
models is in the production cross section and the branch- 
ing fraction to quarks, and we use the same simulated 
event sample for both models. 



contribution, the SM s-channel single top quark contribu- 
tion, and the interference between them. This combined 
cross section has been calculated at leading order using 
CompHEP [13] and then multiplied by the NLO/LO 
cross section ratio from Table VII of Ref. |2j. The fac- 
torization scale has been set equal to the invariant mass 
of the W boson. There is no such interference term for 
right-handed W boson interactions, and the cross sec- 
tions in the two right columns of Table [I] have been taken 
directly from Ref. f2J- For W' R boson interactions, the 
product of production cross section and branching frac- 
tion depends on whether the decay to leptons is allowed 
or not. The branching fraction for the decay W — > tb is 
about 3/12 (3/9) if the W boson decay to quarks and 
leptons (only the decay to quarks) is allowed. The sys- 
tematic uncertainty on the cross section includes com- 
ponents for factorization and renormalization scale, top 
quark mass, and parton distribution functions, and varies 
between about 12% at a mass of 600 GeV and 18% at a 
mass of 800 GeV. 



> 
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FIG. 2: Histogram of the invariant mass of the top-bottom 
quark system at the parton level for different models of W' bo- 
son production. Shown are the SM s-channel distribution, the 
W L — > tb boson distribution, including the interference with 
the SM contribution, and the W' R — > tb boson contribution, 
for a W' boson mass of 600 GeV. 

Figure [3 compares the invariant mass distribution for 
the W models with left-handed coupling (including in- 
terference) and right-handed coupling (no interference) 
with the SM s-channel single top quark distribution. 
While the position and width of the resonance peak at 
600 GeV is not very much affected by the interference, 
there is significant destructive interference for the left- 
handed coupling in the invariant mass region between 
the SM and the resonance peak. 

Table E shows the NLO cross sections for single top 
quark production through a W boson for the three dif- 
ferent models. The cross section for SM-like left-handed 
W boson interactions takes into account the W' L boson 



TABLE I: Production cross section at NLO for a W' boson x 
branching fraction to tb, for three different W' boson models. 
The production cross sections for W' L boson interactions also 
include the SM s-channel contribution as well as the inter- 
ference term between the two. They have been computed at 
leading order and scaled to NLO according to Ref. pj- The 
cross sections for W' R boson interactions differ depending on 
which decays of the W boson are allowed. 

W' mass Cross section x B(W' — ► tb) [pb] 

[GeV] SM+W' L W'g (-> I or q) W' R (-> q only) 



600 
650 
700 
750 
800 



2.17 
1.43 
1.03 
0.76 
0.65 



2.10 
1.25 
0.74 
0.44 
0.26 



2.79 
1.65 
0.97 
0.57 
0.34 



This analysis focuses on the final state topology of sin- 
gle top quark production where the top quark decays 
into a b quark and a SM W boson, which subsequently 
decays leptonically (W — ► ev, fxv; including W — > rv 
with r — > ev, fiv). This gives rise to an event signature 
with a high transverse momentum lepton and significant 
missing transverse energy from the neutrino, in associa- 
tion with two &-quark jets. The largest backgrounds to 
this event signature come from W+jets and tt produc- 
tion. We also consider SM i-channel single top quark 
production as a background in this search. 

The theoretical W boson production cross section 
is more than 15 pb for masses between 200 GeV and 
400 GeV for all three models considered here . The cur- 
rent limits on the single top qua rk p roduction cross sec- 
tion in the s-channel are 6.4 pb (l2l fl^] and 13.6 pb 
and don't depend much on whether the W boson cou- 
pling is left-handed or right-handed. Thus, W boson 
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production with a decay to a top and a bottom quark is 
excluded in this mass region. In this analysis we therefore 
explore the region of even higher masses. 

The analysis utilizes the same dataset, basic event se- 
lection, and background modeling as the D0 single top 
quark search described in Ref. [12|. We select signal- like 
events and separate the data into independent analysis 
sets based on final-state lepton flavor (electron or muon) 
and 6-tag multiplicity (single tagged or double tagged), 
where 6-quark jets are tagged using reconstructed dis- 
placed vertices in the jets. The independent datasets are 
later combined in the final statistical analysis. We per- 
form a binned likelihood analysis on the invariant mass 
distribution of all final state objects to obtain upper cross 
section limits at discrete W mass points. We then com- 
pare these limits to the theoretical prediction and derive 
a lower limit on the mass of the W boson for each of the 
models under consideration. 

The data for this analysis were recorded with the 
D0 detector at the Fermilab Tevatron, a 1.96 TeV 
proton- antiproton collider. The D0 detector has a 
central-tracking system, consisting of a silicon microstrip 
tracker and a central fiber tracker, both located within a 
2 T superconducting solenoidal magnet , with designs 
optimized for tracking and vertexing at pseudorapidities 
|ry| < 3 and \q\ < 2.5 [13], respectively. A liquid-argon 
and uranium calorimeter has a central section covering 
pseudorapidities |tj| ^ 1.1, and two end calorimeters that 
extend coverage to s=s 4.2, with all three housed in sep- 
arate cryostats [2lj . An outer muon system, at \r)\ < 2, 
consists of a layer of tracking detectors and scintillation 
trigger counters in front of 1.8 T iron toroids, followed 
by two similar layers after the toroids |22l |. 

The analysis uses data recorded between August 2002 
and March 2004 (230±15 pb -1 of integrated luminosity). 
The data were collected using a trigger that required an 
electromagnetic energy cluster and a jet in the calorime- 
ter for the electron channel, and a muon and a jet for 
the muon channel. The event selection follows that in 
Ref. 0, except that only events with two or three jets 
are allowed; four-jet events are excluded to reduce the 
background contribution from tt production. 

In the electron channel, candidate events are selected 
by requiring exactly one isolated electron (based on a 
seven-variable likelihood) with transverse energy Et > 
15 GeV and |r?dot| < 1.1. In the muon channel, events 
are selected by requiring exactly one isolated muon with 
transverse momentum px > 15 GeV and | r/dot | < 2.0. For 
both channels, the events are also required to have miss- 
ing transverse energy $t > 15 GeV. Jets are required to 
have Et > 15 GeV and |?7det| < 3.4. Events must have 
exactly two or exactly three jets, with the leading jet ad- 
ditionally required to have Et > 25 GeV and |r?det| < 2.5. 
At least one of the jets is required to be 6-tagged using a 
secondary- vertex algorithm |23( . We separate the dataset 
into orthogonal subsets based on whether one or two jets 



are 6-tagged. 

We estimate the acceptances for W boson produc- 
tion of single top quarks using events generated by the 
CompHEP 4.4.3 matrix element event generator (li| . 
The same program is also used to estimate the yield for 
the SM single top quark background. Interference be- 
tween the SM s-channel and W' L boson production is 
taken into account in the CompHEP event generation 
for left-handed couplings. The W' boson signals are nor- 
malized to the NLO cross section from Table U and we 
use the CTEQ6L1 parton distribution functions [24|. 

We use both Monte Carlo and data to estimate the 
other background yields. The VF+jets and diboson (WW 
and WZ) backgrounds are estimated using simulated 
events generated with ALPGEN [2^. The diboson back- 
ground yield is normalized to NLO cross sections com- 
puted with MCFM |2i|. The fraction of heavy-flavor 
(Wbb) events in the W+jets background is determined 
at the parton level, using MCFM with the same parton- 
level cuts applied as for the samples used in the simu- 
lation. The overall VF+jets yield is normalized to the 
data sample before requiring a 6-tagged jet. This nor- 
malization to data also accounts for smaller contribu- 
tions such as Z+jets events, where one of the leptons 
from the Z boson decay is not reconstructed. The tt 
background is estimated using simulated samples gen- 
erated with ALPGEN, normalized to the (N)NLO cross 
section calculation: a(tt) = 6.7 ± 1.2 pb |27j. The back- 
ground due to SM i-channel single top quark produc- 
tion is normalized to the NLO cross section calculation: 
a(tqb) = 1.98 ± 0.32 pb 0. When investigating the 
right-handed W' boson coupling, the SM s-channel is 
also added as a background. The uncertainty on the top 
quark mass is taken into account in the cross section un- 
certainty The parton-lcvcl samples are then processed 
with PYTHIA 6.2 [51] and a GEANT |29l] -based simulation 
of the D0 detector, and the resulting lepton and jet en- 
ergies are further smeared to reproduce the resolutions 
observed in data. Both the shape and the overall nor- 
malization of the multijet background is estimated from 
data, using multijet data samples that pass all event se- 
lection cuts but fail the electron likelihood requirement in 
the electron channel or the muon isolation requirement in 
the muon channel. The simulated signal and background 
samples include not only the decays W — ► ev, fiv, but also 
the small contribution from W — > rv with r — ► eis, [iv. 

The large mass of the W' boson sets it apart from all 
background processes, hence the best place to look for 
such a particle is the distribution of the reconstructed 
invariant mass in the resonance production process. We 
reconstruct the invariant mass of the W' boson (the in- 
variant mass of all final state objects y/s) by adding the 
four-vectors of all reconstructed final state objects: the 
jets, the lepton, and the neutrino from the W boson de- 
cay from the top quark decay. The ccy-components of the 
neutrino momentum are given by the missing transverse 
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energy. The z-component is calculated using a SM W bo- 
son mass constraint, choosing the solution with smaller 
\p"\ from the two possible solutions. In order to isolate 
the W boson signal, we require y/§ > 400 GeV. 

Figure |3 shows a comparison of the invariant mass dis- 
tribution in data to the sum of all background processes. 
Also shown are the expected contributions for W' bosons 
with left-handed and right-handed couplings at three dif- 
ferent masses. 
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FIG. 3: The reconstructed W' boson invariant mass for sev- 
eral different W' boson masses as well as background pro- 
cesses for (a) left-handed W' boson couplings, and (b) right- 
handed couplings when only the decay to quarks is allowed. 
Electron, muon, single-tagged, and double-tagged events are 
combined. 

The observed event yield is consistent with the back- 
ground model in every bin within uncertainties. There 
arc two events at an invariant mass of more than 
800 GeV, with an expected background of about 0.5 
events. This excess of events is consistent with an up- 
ward fluctuation of the background. 

Systematic uncertainties are evaluated for the simu- 
lated signal and background samples, separately for elec- 



trons and muons and for each 6-tag multiplicity. The 
dominant sources of systematic uncertainty on the signal 
and background acceptances are (a) the uncertainty on 
the 6-tag modeling in the simulation, (b) the uncertainty 
from the jet energy scale, (c) 5% uncertainty on the ob- 
ject identification efficiencies, (d) 5% uncertainty on the 
trigger modeling, and (e) 5% uncertainty on the modeling 
of jet fragmentation |l3| . Each of these systematic un- 
certainties has been evaluated by varying the uncertainty 
for each object in the event (electrons, muons, jets) up 
and down by one standard deviation, and then propa- 
gating the updated objects and corresponding weights 
through the analysis chain. The uncertainty on the in- 
tegrated luminosity is 6.5%. The background yields also 
have uncertainties from the cross sections, which vary 
from 8% for diboson production to 15% for SM t-channel 
single top quark production and 18% for the tt sam- 
ples . Since the VF+jets background is normalized 
to the data before tagging, the yield estimate is mainly 
affected by uncertainties related to 6-tagging. These in- 
clude the 6-tag modeling uncertainty, and the uncertainty 
in the flavor composition before tagging derived from 
MCFM, which is estimated at 25%. The IF+jets back- 
ground yield estimate also has an uncertainty component 
from the parton level modeling of the y/§ distribution, 
which we estimate as 10% based on event yield compar- 
isons in the sample before requiring a 6-tag. The un- 
certainty in the background yield due to the jet energy 
scale varies between 15% and 30% for the single top, 
top pair, and diboson background samples. The uncer- 
tainty is large in these samples because most events have 
a small invariant mass and only very few events are in 
the region y§ > 400 GeV. Changing the jet energy by 
a small amount doesn't change the overall distribution 
very much, but it has a large impact on the number of 
events in the region Vs > 400 GeV. The uncertainty from 
6-tag modeling is about 8% in the single-tagged sample 
and about 20% in the double-tagged one. The total un- 
certainty on the multijet samples is large (s» 35%) due 
to the small number of events in the data sample used to 
model this background. 

Due to their similar kinematic properties, the W bo- 
son signal processes all have very similar systematic un- 
certainties. The overall yield uncertainty due to the 
jet energy scale is small (1-2%) for the signal processes 
because most of the signal events are in the region 
Vs > 400 GeV. The overall yield uncertainty for the 
signal samples has significant contributions from 6-tag 
modeling (4% for the single-tagged, 16% for the double- 
tagged sample) and trigger modeling. The uncertainty 
in the signal region is significantly larger. For example, 
the yield uncertainty due the jet energy scale for a cut 
of Vs > 600 GeV is about 40% for the W' R (600 GeV) 
sample. 

Table [D] shows the event yield in the region \/j > 
400 GeV for all samples, including the total systematic 



uncertainty. The uncertainty includes both acceptance 
and normalization components. 



TABLE II: Event yields with uncertainty after selection, for 
the electron and muon channel, single-tagged and double- 
tagged samples combined, after event selection and requiring 
\/5 > 400 GeV. The W+jets row also includes diboson back- 
grounds. The total uncertainty on the background sum takes 
correlations between different backgrounds into account. 



Event Yields for VI > 400 GeV 
SM+Wj, W' R (-> I or q) W' R (-> q only) 



Signals 






W (600 GeV) 13.0 ±2.3 
W (650 GeV) 7.1 ±1.3 


13.8 ±2.4 


18.4 ±3.2 


7.9 ±1.1 


10.4 ±1.5 


W (700 GeV) 4.4 ±0.8 


4.6 ±0.8 


6.0 ±1.1 


W (750 GeV) 2.4 ±0.4 
W (800 GeV) 1.6 ±0.3 


2.6 ±0.5 


3.4 ±0.6 


1.5 ±0.3 


1.9 ±0.4 


Backgrounds 






SM t-channel 


1.9 ±0.8 




tt 


16.9 ±5.6 




lT±jets 


17.8 ±4.5 




Multijet 


4.4 ±1.5 




Background sum 


41.0 ±10.2 




Data 


30 





The observed data are consistent with the background 
predictions within uncertainties. We therefore set up- 
per limits on the W boson production cross section for 
several different W boson masses in each model. We 
use a Bayesia n ap proach [30| and follow the formalism 
given in Ref. |l2j. The limits are derived from a like- 
lihood function that is proportional to the probability 
to obtain the number of observed counts. Binned likeli- 
hoods are formed based on the final state invariant mass 
distribution, assuming a Poisson distribution for the ob- 
served counts and a flat prior probability for the signal 
cross section. The priors for the signal acceptance and 
the background yields are multivariate Gaussians cen- 
tered on their estimates and described by a covariance 
matrix taking into account correlations across the differ- 
ent sources and bins. 

We combine the electron and muon, single-tagged and 
double-tagged analysis channels. Figure 01 shows the 
cross section limits together with the cross sections from 
Tableland their uncertainties. 

At the 95% confidence level, the shaded areas above the 
solid lines are excluded by this analysis. The intersection 
of the solid line with the lower edge of the uncertainty 
band on the predicted cross section defines the 95% con- 
fidence level lower mass limit for each model. Together 
with the limit from the SM s-channel single top quark 
search we thus exclude the presence of a W boson 
with SM-like left-handed coupling if it has a mass be- 
tween 200 GeV and 610 GeV. We also exclude the pres- 
ence of a W boson with right-handed couplings that is 
allowed to decay to leptons and quarks (only quarks) if 
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FIG. 4: Cross section limits at the 95% confidence level versus 
the mass of the W' boson with (a) left-handed couplings and 
(b) right-handed couplings. Also shown are the NLO cross 
sections according to Table Q and the expected limits. The 
shaded regions above the circles are excluded by this mea- 
surement. 



it has a mass between 200 GeV and 630 GeV (670 GeV). 
This is the first direct search limit for W boson pro- 
duction that takes interference with the SM into account 
properly. It is also the most stringent limit in the top 
quark decay channel of the W boson. 
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